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ABSTRACT 

Methods are described for comparing the educational 
goals of parents and school administrators. Data used to define goals 
consist of statements that parents nade in the course of 
tape-recorded, personal interviews. A list of 450 goal statements was 
then categorized by the same parents and by a sample of school 
administrators. Percentages were then calculated of the parents and 
administrators who placed their statements under each category. 
Although focusing on differences in order to make comparisons, this 
research indicates that agreements were dominant. The second section 
outlines in detail the methodological procedures used in the 
preceding section. Procedures are described with some over-simplified 
"sample" data so that each process can be illustrated. In order to 
cope with the guantitaties of data produced by survey work, some 
available computer programs are also described. (Author/RC) 
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MEASUREMENT AND ANALYSIS TECHNIQUES 
FOR STUDY tNG EDUCATtONAL GOALS 

This paper will discuss a comparison between school administrators and par- 
ents. The primary intent of this comparison is not to conclude what differences, 
exist between these two groups of pi3ople-*that we cannot do because of certain 
sampling and sample size problems — but to illustrate the process of comparison 
using the techniques developed In the Albuquerque goals study. Accordingly, this 
paper will focus predominantly on meithods rather than on findings. 

Before turning to the problems of comparison, let me review briefly the pur-* 
pose of the goals study and how data were gathered. One purpose of this study Is 
to identify areas of potential or actual conflict between various constituent 
groups of the public school system. Existence of conflict, we believe, implies 
the existence of different goals held by the various constituent groups. That 
Is, we expect conflict situations to arise when constituent groups have different 
goals for their school (or perhaps the same goals at different priorities). 

How can one test such a proposition as this? It clearly entails having some 
measurements of "goals,'* however they are defined^ and having some method for col-- 
lecting goals to form an aggregate measure for each constituent group. These are 
difficult requirements both conceptually and practically and we don't pretend to 
have soljved them In the following comparison. What we have done, however, is to 
begin tc define the problems with some actual data and we are surprised and 
pleased,, as we hope you will be, with the reasonableness of the preliminary results < 

The data we have used to define goals consists of statements that parents 

\ 

made in the course of tape-recorded, personal interviews. Every attempt was made 
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to extract as much information as possible in these interviews about all kinds 
of goals for schools — from realistic to Utopian, from rational to unexpectedly 
peculiar. The results of these interviews produced literally thousands of goa'l 
statements. By the way, our parents were sampled to represent relatively low- 
income minority groups many of whom had some difficulty because of inexperience 
in phrasing goal statements. However, they have reported to the interviewers 
that they generally enjoyed the process and some said they had not worked so hard 
since school. Their goal statements, as you will see, are not the typical jargon 
used by some of us. 

Once the statements were typed up, we eliminated redundancies and ambiguities 
while trying to hold on to the original wording insofar as possible. Our list 
of statements is still full of redundancies and ambiguous statements, but we 
feel that we would rather be too lenient with the cutting process than' run the 
risk of ending up with another '"typical" list of goals, such as those produced 
\y\ graduate students' exercises. 

The list of ^50 goal statements was then categorized by the same parents 
who had given us the statements and also by a sample of school adniinistrators who 
were kind enough to spend their time on this project. To understand this process 
of categorization. Imagine a deck of ^50 cards, each with a different goal state** 
ment written on It, which is then sorted Into 37 bins, each one labeled with a 
category name. For example, goal statement 20 which read: 

"(luick learners should receive more help" 
could be placed with other goal statements under a category label such as; 
"Aid the Student who is slow to learn or quick to learn" or 
"Give the student who wants to go to college an adequate foundation 
and provide guidance for those entering college." 
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After all of the goal statements were placed In categories (with a couple 
of extra categories as dumping grounds), we calculated the percentages of the 
parents and of the administrators who placed their statements under each cate- 
gory. For example, the above goal statement about "quick learners" was catego- 
rized as follows: 



CATEGORY PARENTS ADMINISTRATORS 

"Aid the Student..." 66.7* 0.0% 

"Give. . .adequate foundation*.." 16.7* 'lO.O* 



This example shows that in some cases there are disagreements betv/een par- 
ents and administrators on hovi goal statements should be categorized, and Table 
1 lists the statements which show the largest differences. 



TABLE J. Statements Most Differently Categorized By 
Parents A\nd Administrators. 

10. There should be raore seminars on current events operated after school 
hours. 

20. Quick learners should receive more help. 

6^. College-bound students should take courses which would help them 
earn money. 

80. Provisions should be made to teach home-bound students. 
110. Students should be taught to think. 

117. Students should be taught about the social aspect cf life. 

1^^. School should provrde rosvters of technical schools where adviced 

training is available. 
171. Courses should be more comprehensive. 

185. The educational system should meet the needs of the students. 

253. Student should not graduate until they are 18. 

253. Parents should be notified if their child becomes ill. 

2/5. -The Board of Education should have full control of the students. 

281. School should have stronger discipline but should use physical 

or corporal punishment rather than suspension. 
288, If a student is in serious discipline trouble the principle should 

call the parents • 

292. All of students who create discipline problems should be placed 
in a corrective school where the discipline is extremely rigid. 
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295. Students should be allov/ to leave class without permission, 
302. A "hotline" should be established to help students anonymously 

with their problems. 
31 K A room should be provided where students can go if they don't 

want to go to class. 
330. Coaches must realize that they are teachers and educators as 

well as coaches • 

332. Students should be given enought assignments so that they have 

to study to get by. 
338. No student should ever fail. 

hOS* There should be a regular newsletter to the parents to keep them 

informed of school activities. 
kOS. There should be more communication between parents and school 

board. 



It is difficult to interpret these differences, however, because the statements 
In Table I are of such different types and represent such different values that 
they cannot be treated equally. 

We can remedy part of this interpretation problem because we have some in-^ 
formation about which statements the respondents value. Prior to categorization 
we asked all respondents to sort the goal statements into three preference cate- 
gories: like, dislike, and no preference. V/ith this information we can select 
just thoiie statements which respondents value and base the comparison on these 
statements. 

Table 2 lists the statements and the proportion of the time they were "liked." 
For comparative purposes, v/e will look only at those statements which were liked ... 
by 75% or more of both groups. 



TABLE 2. Valued Statements (percent categorized as "Like,") 



STATEMENT NO. 



ADMINISTRATORS {%) 



PARENTS i%) 



10 



52 



20 



35 



43 



64 



US 



71 
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80 


95* 




81* 


110 


95* 




86* 


117 


90* 


-X- 


62 


1H 


95* 




86* 


171 


30 


-X- 


86* 


185 


95* 




86* 


253 


0 




10 


259 


95* 




100* 


275 


10 




38 


281 


0 




29 


288 


95* 


- 


100* 


292 


20 




29 


295 


5 




5 


302 


25 




57 


311 


15 




19 


330 


100* 




81* 


332 


ko 

m 




52 


338 


5 




33 


^»05 


90* 




90* 


409 


75* 




81* 




(N = 20) 




(N 



There are nine statements which fall into 75^ or more "liked," as shown 
above in Table 2 by asterisks. 
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TABLE 3- Comparison On Categorization Of Goal Statements. 



STATEMENT 



80 



% OF 



110 



185 



259 



288 



330 



ADMIN: 
PARENT: 

ADMIN: 

PARENT: 

ADMIN: 

PARENT: 

ADMIN: 

PARENT: 

ADMIN: 

PARENT: 

ADMIN: 
PARENT: 

ADMIN: 
PARENT: 



ERIC 



25(3) 

^(2) 

21(16) 

5(A) 

15(13) 

8(3) 

k (19) 

30(2) 

67(2) 

5(2) 

5(16) 

8(3) 

13(12) 

50(25) 

5(37) 

A(21) 

50(33) 

20(2A) 

A(22) 

17(33) 

15(23) 

10C31) 

A (22) 

17(31) 



Mio) 



RESPONDENTS IN (CATEGORY) 

30(6) A5(16) 

8(3) '♦(5) 21(6) m) 

8(17) 25(19) 

5(5) A5(6) 10(8) 5(12) 

5(18) 10(19) 

A2(5) 25(6) f»(10) 17(13) 



70(IA) 

13(6) A(11) 13(1A) 
5(3) 10(5) 15(6) 25(8) -5(12) 
5(17) 15(18) 10(19) 
13(5) 8(6) M8). 8(10) ^»(ll) 
25(1A) A(18) 13(19) 
10(26) 25(29) 5(33) 5(35) 

A(23) A(2A) 25(25) A(29) 
8(35) 
30(25) 
17(2if) 

21(35) 
5(2A) 

5(37) 
^6(23) 
17(37) 



5(26) 
8(25) 

A(36) 
5(25) 



A5(35) 
13(29) 

13(37) 
50(27) 



A (30) 



10(28) 



A(25) 8(2;) ki.23) 
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ADMIN: 5(25) 85(29) 5(30) 5(33) 

^05 PARENT: 4(21) 4(22) 8(25) 29(29) 4(30) 

4(31) 21 (33) 17(35) 8(37) 

ADMiN: 5(26) 75 (2?) 5(30) 10(33) 5(37) 
409 PARENT: 29(35) 29(29) 25(33) 4(35) 

13(37) ... 

In Table 3» the symbols represent tHe percent of the comparison groups that 
placed the statement in the category identified by parentheses. Thus 21(5) means 
that 21% put this statement in category #5- Table 4 lists the'cat^gbries 
refered to in this comparison. 

TABLE 4. Category Definition By Number. 

1. Provide an opportunity to participate in competitive athletics, 

2. Provide vocational training. 

3* Aid the student who is slow to learn or quick to learn. 

4. Develop a sense of decency; right and wrong; and honesty, 

5. Develop self -awareness, individuality, and a purpose for livJrig one's life. 

6. Develop the students' basic academic skills. 

7. Provide an opportunity for development of physical fitness, 

8. Prepare students to function in society as a citizen. 

9. Develop an appreciation of one's own cultural heritage. 

10. Develop tolerance and respect for others, regardless of their ethnic origin, 
religion, sex, or age. 

11. Give the student who v/ants to go to college an adequate foundation and provide 
guidance for those entering college. 

12. Develop talents, interests, and hobbies which students can enjoy in adult 
life. 
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13. Develop the ability to express one*s thoughts clearly through the written 
or spoken word and develop the ability to comprehend the v/ritten or spoken 
words of others. 

]k. Expose students to opportunities for careers and guide them in »-aking a 
career choice. 

15. Develop an appreciation of the history and philosophy of religion. 

16. Provide aid to the student who is emotionally handicapped. 

17. Prepare students for the ?: les of spouse, parent, or single adult. 

18. Prepare students to manage the practical aspects of adult life, 
19* Spare 

20. Community use of school facilities. 

21. Student choice of courses and activities. 

22. Personality of teachers, 

23. Learned skills and knowledge of teachers, 
2k. Discipline by formal authority, 

25. Efficient administration of the school. 

26. Food service. 

27. Training and evaluation of teachers. 

28. Academic standards. 

29. School -community Interaction. 

30. Discipline by guidance and training. 

31. Teacher Interaction with students in the classroom. 

32. Facilities and equipment for students' education (libraries, buildings, 
texts, classrooms, etc.) 

•33. Parent-teacher interaction. 

3k. Keeping students off the streets and off the parents' hands. 

35. Discipline by parents and parent-teacher communication, 

36. Counseling services for students — such as testing and scheduling. 
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37. Spare 

The entries of Table 3 have been underlined to help identify the greatest 
disparities in categorization (about 20% or more). Thus, using statement 110 
as an example (''students should be taught to think") > we see that parents tend 
more than administrators to classify this under: 

"Develop self-awareness, individuality, and a purpose for 

one's life." 

Whereas, administrators tend more than parents to classify this statement under: 

"Develop the students' basic academic skills." 
Just a moment's reflection shows how some misunderstandings could arise as the 
two groups conceived of programs to help students learn to think. 

However, these tables raise as many questions as they answer. For example, 
why do only S% of the school administrators and 33^ of the parents "like" the 
statement "no student should ever fail?" Why do 86^ of the parents and only 
30% of the administrators sampled "like" the statement "courses should be more 
comprehens i ve?" 

One of the challenges ahead of us is to design a sampling frame that will 
give us enough data to make accurate estimates of these group differences. 
Another challenge is to structure our responses so that we can look more deeply 
into the sources of the differences between parents and administrators. Are 
they due to differences in language or to real differences in expectations? 

In closing it is worth noting that although this paper has focused on 
differences to be able to make comparisons, the dominant finding is that there 
are mostly agreements — at least in Albuquerque; at least in ouir sample. We 
find, for example, that when parents and administrators rank categories (from 
O 6 best to like least), there is an overall correlation of about 0.8 between 



the two lists. Further, we should not forget that the differences discussed in 
this paper could only produce nine statements out of kSO where there were clear 
differences. We can only hope that these agreements rest on true similarities 
in expectations and do not result from mere language. 
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Technt cal Discussion 

The following section outlines in detail the methodological procedures 
used In the first part of this paper. The procedures are described with 
seme oversimplified '^sample'* data so that each process can be illustrated. 
In order to cope with the quantities of data produced by survey woric some 
available computer programs are also described. 

At the outset one needs to note that we have not '^measured'* respondents" 
goals for schools. There are characteristics of people's goals that are not 
easily, and perhaps not possibly, reducible to numeric properties. These 
are the personal subjective characteristics that are more suitably summarized 
by journalistic efforts than by numerical indices. 

Statistics has been some help, however, in summarizing or enumerating 
the quantitative aspects of goals, such as how many times goal statements 
are categorized differently, which categories are most popular^ etc. fn 
short, we have counted rather than measured aspects of people's goals for 
schools. 

This approach has introduced special problems in the comparisons 
between groups of respondents. As this v;orl< progresses, these analytic 
problems will receive more attention. The following woric represents only 
the most preliminary worl<. 
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Aggregating the data; The Distribution Matrix 

The first step fn the analysis is to create the distribution matrix. 
This matrix consists of the raw data aggregated over all respondents In 
a particular group (e.g., parents). The matrix shows how many respondents 
put each of the goal statements into the possible category boxes. 

For example, if there were k respondents (parents), 3 categories 
and 5 goals statements, the raw data might looI< lil<e Figure 1. 
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CATEGORIES (3) 



RESPONDENTS {k) 
FIGURE 1. RAW DATA FOR FOUR RESPONDENTS 
In this case, the distribution matrix v;ou)cl be the sun of the raw data 
matrices and would produce Figure 2. 
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FIGURE 2. DISTRIBUTION MATRIX (RAW AGGREGATE SCORE) 
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To interpret the distribution matrix, first look at row 3. All ^ 
respondents put this statement in category 1. It seems reasonable to 
assume that this goal statement exemplifies or is a good operationaUsm 
of category 1. So. too with goal statement 1 (row 1), yet here one 
respondent disagreed and we may want to weight this statement somewhat 
less than statement 3 to indicate that we are not on as firm ground. 

One way of representing the distribution matrix that gives an 
intuitively clear feeling of the homogeneity of response agreement is to 
let the cells represent percent agreement on categorical location. For 
example, the distribution matrix of Figure 2 would produce the matrix of 
Figure 3, 
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75 


25 


0 
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50 


50 


0 


3 


100 


0 


0 




25 


50 


25 


5 


25 


0 


75 



FIGURE 3. DISTRIBUTION MATRIX (PERCENT AGGREGATE SCORE) 
which illustrates the clear agreement on statement 3 and the less firm but 
predominant agreement on statements 1 and 5. Statements 2 and k might be 
worth looking at to see if there are ambiguities or connotations that 
confuse respondents. If not, they may be useful as general, if vagie, 
goal statements or they may indicate the absence of a category name. They 
may also be statements upon which an otherwise homogeneous group (parents) 
simply do not agree. 



The Definttton Problem 

Even with the distribution matrix In percent aggregate form, there 
remains a question as to hov; well we have summarized the goals of the 
respondents. The problem here is a definitional problem. 

If we choose to define a group of respondents in full detail, 
considering every possible categorization that might represent valid 
individual conceptualizations of goals, then the above distribution 
matrix does an adequate job. The percent aggregate matrix gives an over- 
all picture (the 75%-plus cells) and incorporates the minority view and 
noise In the data as well (2f)%-minus cells). We will later want to 
consider the Implications of not being able to separate minority views 
from error and ambiguity. 

On the other hand, if v/e feel that the goals of a group of respondents 
are of significance only insofar as they clearly represent consensus and 
agreement within the group, then the distribution matrix contains large 
amounts of useless Information. A more appropriate aggregated data 
matrix might be as shown in Figure k where the only data shown is that 
which exemplifies clear -agreement. 
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FIGURE k: SUMMARY DISTRIBUTION MATRIX 
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We win not use this summary matrix explicitly, but it represents 
an easy summarizing technique v^hich v/lll be useful later. It also 
represents a logical assumption which v/UI be used to screen data In 
some of the computer progroms that follow. 

Comparing Groups of Respondents; The Difference Matrix 

Once we have summarized a group of respondents in a composite 
distribution matrix, we will at some point want to be able to say 
whether this group of respondents is the same as, or different than, 
another group,. This is the comparison problem, and It Is such an 
intractable problem that we will have to make some assumptions to even 
get started on I t. 

Suppose that there are two distribution matrices, each representing 
a different respondent group, say parents and school administrators. One 
conceptually clear way of expressing difference or 1 fineness Is by sub- 
traction, cell-bycel 1 . The resulting matrix (called the difference 
matrix) contains all of the information about group differences and 
Information about where these differences exist. Figure 5 summarizes the 
two distribution matrices and shows the resulting difference matrix. 



Old Distribution Matrix 
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75 



New Distribution Matrix 
"Administrators" 
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50 
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50 


50 


50 
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25 


25 


50 


25 


75 


0 
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RESULTING DIFFERENCE MTRIX 



25 


25 
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(50) 


0 


50 


50 


(100) 


50 


50 


0 


(100) 


0 


25 


25 


(50) 


0 


75 


75 


(150) 



(scores) 

FIGURE 5. DIFFERENCE MATRIX (cont'd.) 

The difference matrix contains enough Information to answer several 
comparative questions concerning the two groups. For example, using 
the difference matrix alone, one could find which goal statements v/ere 
differently categorized by each of the two groups. To show how this 
question would be answered, we will use the example In Figure 5 with a 
reminder that the real difference matrix v;ould be A50 statements long 
by 37 categories wide (16,650 cells). Because of the size of real 
difference matrix, the sample will be the last difference matrix to 
appear in print.' While "using it to answer some comparative questions, 
" we wilt find a way to sunrjrarize the information It contains in more 
manageable form. 

Because the sample matrices In Figure 5 are so snail, a few moDents 
study will show that only In one case (statement 5) did the two groups 
really differ In the categorization of a well-defined statement. There 
are some real differences in statements 2 and 3. but In both of these 
cases, there was enough individual variation (or noise) within one or 
O toth of the groups to make a comparison between categorizations difficult. 
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The difference matrix flags each statement where there Is both clear 
intragroup definition and Intergroup disagreement by having the highest 
scores along the rov/ for that statement — this makes the statement easy 
to find and to Interpret. 

The difference matrix does not perform as well on the task of 
discovering which statements are similarly categorized, however. Both 
statement 1, which Is the only case where the two groups agreed on a 
well-defined statement, and statement A have the same row data. Yet 
statement 4 Is the most poorly defined of all the statements In terms 
of being Inconsistently categorized v/ithin a particular group. Vfhy 
should two statements so differently categorized give rise to the same 
row data In the difference matrix? 

The answer illustrates a weakness of the difference matrix. Be- 
cause the difference matrix Is constructed by simple subtraction, it 
gives the same results for the difference between two large numbers or 
for the difference between several small numbers. By Itself, the 
difference matrix does not allow the tv;o cases to be distinguished. On 
the other hand, the matrix Is useful In detecting well-defined and 
dissimilarly categorized statements because this situation produces a 

unique case of finding the difference between a large and a small number. 

i 

jTable 1 summarizes this situation and illustrates some Intermediate 

t 

cases (statements 2 and 3)* This table also shows the vector of total 

I 

scores for each statement (In parentheses next to the difference matrix 

I 

In Figure 5) which can be used as a summary device without further loss of 
Infornjatlon. For notatlonal convenience, this vector of scores Is referred 
to as the difference vector. 
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Parents 


Comparison 


Adminl stra tors* 


Vector 


1 


Well defined 


(good agreement) 


very wel 1 defined 


• 

50 


2 


Poorly defined 


(medium agreement) 


poorly defined 


100 


3 


Very wel 1 defined 


(medium agreement) 


poorly defined 


100 


k 


Very poorly defined 


(good agreement) 


very poorly defined 


50 


5 


Well defined 


(poor agreement) 


wel 1 defined 


150 



Key: Column 1: Statement number 

Column 2 and k: Intragroup definitional agreement 

Column 3: Intergroup agreement on categorization 

Column 5« Row totals of the difference matrix 



TA3LE 1. SUMMARY OF FIGURE 5 MATRICES 
(AND THE DIFFERENCE VECTOR) 

Table 1 finally illustrates the pattern necessary for the analysis of 
real data. Notice that the largest elements of the difference vector iden- 
tify those statements which have a relatively consistent categorization 
pattern within a group of respondents (good definition) with a relatively 
different pattern between groups (poor agreement). Such a pattern is 
necessary for simplifying the task of comparing two ^^50 X 37 distribution 
matrices — although other patterns could be equally useful. This difference 

vector has the virtue th^t it calls attention to groups disagreement, which 

j 

has cljear and useful policy inpHcations for the data to be analyzed. 

The analysis which follcrv/s was done on the University of New Mexico 
computer. The programs are in FORTRAN and are available for the asking 
from tLe National Institute of Education. A brief summary of the programs 
might i>e useful at this point to illustrate the logical operations. 
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(NTERVU (program name) — an Interactive program to create the data 
file for each respondent. The file contains the raw data of 
Figure 1. 

GROUP--C0I lects Individual respondent data files to form the group 
distribution matrices (raw and percent) of figures 2 and 3* 

COHPAR— col lects the output files produc^id by GROUP and creates 
the difference matrix and difference vector for two groups of 
respondents. 

CHECK — essentially similar to COMPAR, this program searches for 

the largest elements of the difference vector and locates statements 

in the ''extra" categories (I9 and 37). 

PREFER — creates individual respondent data files from preference 
sorting of each statement Into 'Mike, dislike, neutral" categories. 



